Explore your experimental designs and theories before you exploit them!

In many areas of the social and behavioral sciences, the nature of the experiments and theories that best capture the underlying constructs are themselves areas of active inquiry. Integrative experiment design risks being prematurely exploitative, hindering exploration of experimental paradigms and of diverse theoretical accounts for target phenomena.

Enhancement of mediodorsal thalamus rescues aberrant belief dynamics in a mouse model with schizophrenia-associated mutation.

Optimizing behavioral strategy requires belief updating based on new evidence, a process that engages higher cognition. In schizophrenia, aberrant belief dynamics may lead to psychosis, but the mechanisms underlying this process are unknown, in part, due to lack of appropriate animal models and behavior readouts. Here, we address this challenge by taking two synergistic approaches. First, we generate a mouse model bearing patient-derived point mutation in Grin2a (Grin2aY700X+/-), a gene that confers high-risk for schizophrenia and recently identified by large-scale exome sequencing. Second, we develop a computationally trackable foraging task, in which mice form and update belief-driven strategies in a dynamic environment. We found that Grin2aY700X+/- mice perform less optimally than their wild-type (WT) littermates, showing unstable behavioral states and a slower belief update rate. Using functional ultrasound imaging, we identified the mediodorsal (MD) thalamus as hypofunctional in Grin2aY700X+/- mice, and in vivo task recordings showed that MD neurons encoded dynamic values and behavioral states in WT mice. Optogenetic inhibition of MD neurons in WT mice phenocopied Grin2aY700X+/- mice, and enhancing MD activity rescued task deficits in Grin2aY700X+/- mice. Together, our study identifies the MD thalamus as a key node for schizophrenia-relevant cognitive dysfunction, and a potential target for future therapeutics.

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia.

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits.

Prefrontal cortex state representations shape human credit assignment.

People learn adaptively from feedback, but the rate of such learning differs drastically across individuals and contexts. Here, we examine whether this variability reflects differences in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative reward learning task, we link the specificity of credit assignment-how well people are able to appropriately attribute outcomes to their causes-to the precision of neural codes in the prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared vto nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in concert to match the neural codes from feedback to those at choice, and the strength of these common neural codes predicts credit assignment precision. Together this work provides a window into how neural representations drive adaptive learning.

Computational mechanisms of belief updating in relation to psychotic-like experiences.

Introduction: Psychotic-like experiences (PLEs) may occur due to changes in weighting prior beliefs and new evidence in the belief updating process. It is still unclear whether the acquisition or integration of stable beliefs is altered, and whether such alteration depends on the level of environmental and belief precision, which reflects the associated uncertainty. This motivated us to investigate uncertainty-related dynamics of belief updating in relation to PLEs using an online study design. Methods: We selected a sample (n = 300) of participants who performed a belief updating task with sudden change points and provided self-report questionnaires for PLEs. The task required participants to observe bags dropping from a hidden helicopter, infer its position, and dynamically update their belief about the helicopter's position. Participants could optimize performance by adjusting learning rates according to inferred belief uncertainty (inverse prior precision) and the probability of environmental change points. We used a normative learning model to examine the relationship between adherence to specific model parameters and PLEs. Results: PLEs were linked to lower accuracy in tracking the outcome (helicopter location) (ß = 0.26 ± 0.11, p = 0.018) and to a smaller increase of belief precision across observations after a change point (ß = -0.003 ± 0.0007, p < 0.001). PLEs were related to slower belief updating when participants encountered large prediction errors (ß = -0.03 ± 0.009, p = 0.001). Computational modeling suggested that PLEs were associated with reduced overall belief updating in response to prediction errors (ßPE = -1.00 ± 0.45, p = 0.028) and reduced modulation of updating at inferred environmental change points (ßCPP = -0.84 ± 0.38, p = 0.023). Discussion: We conclude that PLEs are associated with altered dynamics of belief updating. These findings support the idea that the process of balancing prior belief and new evidence, as a function of environmental uncertainty, is altered in PLEs, which may contribute to the development of delusions. Specifically, slower learning after large prediction errors in people with high PLEs may result in rigid beliefs. Disregarding environmental change points may limit the flexibility to establish new beliefs in the face of contradictory evidence. The present study fosters a deeper understanding of inferential belief updating mechanisms underlying PLEs.

Age-related differences in prefrontal glutamate are associated with increased working memory decay that gives the appearance of learning deficits.

The ability to use past experience to effectively guide decision-making declines in older adulthood. Such declines have been theorized to emerge from either impairments of striatal reinforcement learning systems (RL) or impairments of recurrent networks in prefrontal and parietal cortex that support working memory (WM). Distinguishing between these hypotheses has been challenging because either RL or WM could be used to facilitate successful decision-making in typical laboratory tasks. Here we investigated the neurocomputational correlates of age-related decision-making deficits using an RL-WM task to disentangle these mechanisms, a computational model to quantify them, and magnetic resonance spectroscopy to link them to their molecular bases. Our results reveal that task performance is worse in older age, in a manner best explained by working memory deficits, as might be expected if cortical recurrent networks were unable to sustain persistent activity across multiple trials. Consistent with this, we show that older adults had lower levels of prefrontal glutamate, the excitatory neurotransmitter thought to support persistent activity, compared to younger adults. Individuals with the lowest prefrontal glutamate levels displayed the greatest impairments in working memory after controlling for other anatomical and metabolic factors. Together, our results suggest that lower levels of prefrontal glutamate may contribute to failures of working memory systems and impaired decision-making in older adulthood.

Adaptive Learning through Temporal Dynamics of State Representation.

People adjust their learning rate rationally according to local environmental statistics and calibrate such adjustments based on the broader statistical context. To date, no theory has captured the observed range of adaptive learning behaviors or the complexity of its neural correlates. Here, we attempt to do so using a neural network model that learns to map an internal context representation onto a behavioral response via supervised learning. The network shifts its internal context on receiving supervised signals that are mismatched to its output, thereby changing the "state" to which feedback is associated. A key feature of the model is that such state transitions can either increase learning or decrease learning depending on the duration over which the new state is maintained. Sustained state transitions that occur after changepoints facilitate faster learning and mimic network reset phenomena observed in the brain during rapid learning. In contrast, state transitions after one-off outlier events are short lived, thereby limiting the impact of outlying observations on future behavior. State transitions in our model provide the first mechanistic interpretation for bidirectional learning signals, such as the P300, that relate to learning differentially according to the source of surprising events and may also shed light on discrepant observations regarding the relationship between transient pupil dilations and learning. Together, our results demonstrate that dynamic latent state representations can afford normative inference and provide a coherent framework for understanding neural signatures of adaptive learning across different statistical environments.SIGNIFICANCE STATEMENT How humans adjust their sensitivity to new information in a changing world has remained largely an open question. Bridging insights from normative accounts of adaptive learning and theories of latent state representation, here we propose a feedforward neural network model that adjusts its learning rate online by controlling the speed of transitioning its internal state representations. Our model proposes a mechanistic framework for explaining learning under different statistical contexts, explains previously observed behavior and brain signals, and makes testable predictions for future experimental studies.

Latent motives guide structure learning during adaptive social choice.

Predicting the behaviour of others is an essential part of social cognition. Despite its ubiquity, social prediction poses a poorly understood generalization problem: we cannot assume that others will repeat past behaviour in new settings or that their future actions are entirely unrelated to the past. We demonstrate that humans solve this challenge using a structure learning mechanism that uncovers other people's latent, unobservable motives, such as greed and risk aversion. In four studies, participants (N = 501) predicted other players' decisions across four economic games, each with different social tensions (for example, Prisoner's Dilemma and Stag Hunt). Participants achieved accurate social prediction by learning the stable motivational structure underlying a player's changing actions across games. This motive-based abstraction enabled participants to attend to information diagnostic of the player's next move and disregard irrelevant contextual cues. Participants who successfully learned another's motives were more strategic in a subsequent competitive interaction with that player in entirely new contexts, reflecting that social structure learning supports adaptive social behaviour.

The computational challenge of social learning.

The complex reward structure of the social world and the uncertainty endemic to social contexts poses a challenge for modeling. For example, during social interactions, the actions of one person influence the internal states of another. These social dependencies make it difficult to formalize social learning problems in a mathematically tractable way. While it is tempting to dispense with these complexities, they are a defining feature of social life. Because the structure of social interactions challenges the simplifying assumptions often made in models, they make an ideal testbed for computational models of cognition. By adopting a framework that embeds existing social knowledge into the model, we can go beyond explaining behaviors in laboratory tasks to explaining those observed in the wild.

Neural connectome prospectively encodes the risk of post-traumatic stress disorder (PTSD) symptom during the COVID-19 pandemic.

BACKGROUND: The novel coronavirus (COVID-19) pandemic has affected humans worldwide and led to unprecedented stress and mortality. Detrimental effects of the pandemic on mental health, including risk of post-traumatic stress disorder (PTSD), have become an increasing concern. The identification of prospective neurobiological vulnerability markers for developing PTSD symptom during the pandemic is thus of high importance. METHODS: Before the COVID-19 outbreak (September 20, 2019-January 11, 2020), some healthy participants underwent resting-state functional connectivity MRI (rs-fcMRI) acquisition. We assessed the PTSD symptomology of these individuals during the peak of COVID-19 pandemic (February 21, 2020-February 28, 2020) in China. This pseudo-prospective cohort design allowed us to test whether the pre-pandemic neural connectome status could predict the risk of developing PTSD symptom during the pandemic. RESULTS: A total of 5.60% of participants (n = 42) were identified as being high-risk to develop PTSD symptom and 12.00% (n = 90) exhibited critical levels of PTSD symptoms during the COVID-19 pandemic. Pre-pandemic measures of functional connectivity (the neural connectome) prospectively classified those with heightened risk to develop PTSD symptom from matched controls (Accuracy = 76.19%, Sensitivity = 80.95%, Specificity = 71.43%). The trained classifier generalized to an independent sample. Continuous prediction models revealed that the same connectome could accurately predict the severity of PTSD symptoms within individuals (r 2 = 0.31p<.0). CONCLUSIONS: This study confirms COVID-19 break as a crucial stressor to bring risks developing PTSD symptom and demonstrates that brain functional markers can prospectively identify individuals at risk to develop PTSD symptom.

Dynamic Representation of the Subjective Value of Information.

To improve future decisions, people should seek information based on the value of information (VOI), which depends on the current evidence and the reward structure of the upcoming decision. When additional evidence is supplied, people should update the VOI to adjust subsequent information seeking, but the neurocognitive mechanisms of this updating process remain unknown. We used a modified beads task to examine how the VOI is represented and updated in the human brain of both sexes. We theoretically derived, and empirically verified, a normative prediction that the VOI depends on decision evidence and is biased by reward asymmetry. Using fMRI, we found that the subjective VOI is represented in the right dorsolateral prefrontal cortex (DLPFC). Critically, this VOI representation was updated when additional evidence was supplied, showing that the DLPFC dynamically tracks the up-to-date VOI over time. These results provide new insights into how humans adaptively seek information in the service of decision-making.SIGNIFICANCE STATEMENT For adaptive decision-making, people should seek information based on what they currently know and the extent to which additional information could improve the decision outcome, formalized as the VOI. Doing so requires dynamic updating of VOI according to outcome values and newly arriving evidence. We formalize these principles using a normative model and show that information seeking in people adheres to them. Using fMRI, we show that the underlying subjective VOI is represented in the dorsolateral prefrontal cortex and, critically, that it is updated in real time according to newly arriving evidence. Our results reveal the computational and neural dynamics through which evidence and values are combined to inform constantly evolving information-seeking decisions.

Noise Correlations for Faster and More Robust Learning.

Distributed population codes are ubiquitous in the brain and pose a challenge to downstream neurons that must learn an appropriate readout. Here we explore the possibility that this learning problem is simplified through inductive biases implemented by stimulus-independent noise correlations that constrain learning to task-relevant dimensions. We test this idea in a set of neural networks that learn to perform a perceptual discrimination task. Correlations among similarly tuned units were manipulated independently of an overall population signal-to-noise ratio to test how the format of stored information affects learning. Higher noise correlations among similarly tuned units led to faster and more robust learning, favoring homogenous weights assigned to neurons within a functionally similar pool, and could emerge through Hebbian learning. When multiple discriminations were learned simultaneously, noise correlations across relevant feature dimensions sped learning, whereas those across irrelevant feature dimensions slowed it. Our results complement the existing theory on noise correlations by demonstrating that when such correlations are produced without significant degradation of the signal-to-noise ratio, they can improve the speed of readout learning by constraining it to appropriate dimensions.SIGNIFICANCE STATEMENT Positive noise correlations between similarly tuned neurons theoretically reduce the representational capacity of the brain, yet they are commonly observed, emerge dynamically in complex tasks, and persist even in well-trained animals. Here we show that such correlations, when embedded in a neural population with a fixed signal-to-noise ratio, can improve the speed and robustness with which an appropriate readout is learned. In a simple discrimination task such correlations can emerge naturally through Hebbian learning. In more complex tasks that require multiple discriminations, correlations between neurons that similarly encode the task-relevant feature improve learning by constraining it to the appropriate task dimension.

Adaptive learning is structure learning in time.

People use information flexibly. They often combine multiple sources of relevant information over time in order to inform decisions with little or no interference from intervening irrelevant sources. They adjust the degree to which they use new information over time rationally in accordance with environmental statistics and their own uncertainty. They can even use information gained in one situation to solve a problem in a very different one. Learning flexibly rests on the ability to infer the context at a given time, and therefore knowing which pieces of information to combine and which to separate. We review the psychological and neural mechanisms behind adaptive learning and structure learning to outline how people pool together relevant information, demarcate contexts, prevent interference between information collected in different contexts, and transfer information from one context to another. By examining all of these processes through the lens of optimal inference we bridge concepts from multiple fields to provide a unified multi-system view of how the brain exploits structure in time to optimize learning.

Response-based outcome predictions and confidence regulate feedback processing and learning.

Influential theories emphasize the importance of predictions in learning: we learn from feedback to the extent that it is surprising, and thus conveys new information. Here, we explore the hypothesis that surprise depends not only on comparing current events to past experience, but also on online evaluation of performance via internal monitoring. Specifically, we propose that people leverage insights from response-based performance monitoring - outcome predictions and confidence - to control learning from feedback. In line with predictions from a Bayesian inference model, we find that people who are better at calibrating their confidence to the precision of their outcome predictions learn more quickly. Further in line with our proposal, EEG signatures of feedback processing are sensitive to the accuracy of, and confidence in, post-response outcome predictions. Taken together, our results suggest that online predictions and confidence serve to calibrate neural error signals to improve the efficiency of learning.

All or nothing belief updating in patients with schizophrenia reduces precision and flexibility of beliefs.

Schizophrenia is characterized by abnormal perceptions and beliefs, but the computational mechanisms through which these abnormalities emerge remain unclear. One prominent hypothesis asserts that such abnormalities result from overly precise representations of prior knowledge, which in turn lead beliefs to become insensitive to feedback. In contrast, another prominent hypothesis asserts that such abnormalities result from a tendency to interpret prediction errors as indicating meaningful change, leading to the assignment of aberrant salience to noisy or misleading information. Here we examine behaviour of patients and control subjects in a behavioural paradigm capable of adjudicating between these competing hypotheses and characterizing belief updates directly on individual trials. We show that patients are more prone to completely ignoring new information and perseverating on previous responses, but when they do update, tend to do so completely. This updating strategy limits the integration of information over time, reducing both the flexibility and precision of beliefs and provides a potential explanation for how patients could simultaneously show over-sensitivity and under-sensitivity to feedback in different paradigms.

The stability flexibility tradeoff and the dark side of detail.

Learning in dynamic environments requires integrating over stable fluctuations to minimize the impact of noise (stability) but rapidly responding in the face of fundamental changes (flexibility). Achieving one of these goals often requires sacrificing the other to some degree, producing a stability-flexibility tradeoff. Individuals navigate this tradeoff in different ways; some people learn rapidly (emphasizing flexibility) and others rely more heavily on historical information (emphasizing stability). Despite the prominence of such individual differences in learning tasks, the degree to which they relate to broader characteristics of real-world behavior or pathologies has not been well explored. We relate individual differences in learning behavior to self-report measures thought to capture collectively the characteristics of the Autism spectrum. We show that young adults who learn most slowly tend to integrate more effective samples into their beliefs about the world making them more robust to noise (more stability) but are more likely to integrate information from previous contexts (less flexibility). We show that individuals who report paying more attention to detail tend to use high flexibility and low stability information processing strategies. We demonstrate the robustness of this inverse relationship between attention to detail and formation of stable beliefs in a heterogeneous population of children that includes a high proportion of Autism diagnoses. Together, our results highlight that attention to detail reflects an information processing policy that comes with a substantial downside, namely the ability to integrate data to overcome environmental noise.

Age-related changes in the functional integrity of the phasic alerting system: a pupillometric investigation.

Enhanced processing following a warning cue is thought to be mediated by a phasic alerting response involving the locus coeruleus-noradrenergic (LC-NA) system. We examined the effect of aging on phasic alerting using pupil dilation as a marker of LC-NA activity in conjunction with a novel assessment of task-evoked pupil dilation. While both young and older adults displayed behavioral and pupillary alerting effects, reflected in decreased RT and increased pupillary response under high (tone) versus low (no tone) alerting conditions, older adults displayed a weaker pupillary response that benefited more from the alerting tone. The strong association between dilation and speed displayed by older adults in both alerting conditions was reduced in young adults in the high alerting condition, suggesting that in young (but not older) adults the tone conferred relatively little behavioral benefit beyond that provided by the alerting effect elicited by the target. These findings suggest a functioning but deficient LC-NA alerting system in older adults, and help reconcile previous results concerning the effects of aging on phasic alerting.

Functional brain network reconfiguration during learning in a dynamic environment.

When learning about dynamic and uncertain environments, people should update their beliefs most strongly when new evidence is most informative, such as when the environment undergoes a surprising change or existing beliefs are highly uncertain. Here we show that modulations of surprise and uncertainty are encoded in a particular, temporally dynamic pattern of whole-brain functional connectivity, and this encoding is enhanced in individuals that adapt their learning dynamics more appropriately in response to these factors. The key feature of this whole-brain pattern of functional connectivity is stronger connectivity, or functional integration, between the fronto-parietal and other functional systems. Our results provide new insights regarding the association between dynamic adjustments in learning and dynamic, large-scale changes in functional connectivity across the brain.

Statistical context dictates the relationship between feedback-related EEG signals and learning.

Learning should be adjusted according to the surprise associated with observed outcomes but calibrated according to statistical context. For example, when occasional changepoints are expected, surprising outcomes should be weighted heavily to speed learning. In contrast, when uninformative outliers are expected to occur occasionally, surprising outcomes should be less influential. Here we dissociate surprising outcomes from the degree to which they demand learning using a predictive inference task and computational modeling. We show that the P300, a stimulus-locked electrophysiological response previously associated with adjustments in learning behavior, does so conditionally on the source of surprise. Larger P300 signals predicted greater learning in a changing context, but less learning in a context where surprise was indicative of a one-off outlier (oddball). Our results suggest that the P300 provides a surprise signal that is interpreted by downstream learning processes differentially according to statistical context in order to appropriately calibrate learning across complex environments.